User interface for medical image review workstation

ABSTRACT

Methods, systems and computer program products for controlling display of different types of medical images and providing touchscreen interfaces for display on a mobile communication device and associated with different image types, e.g., different imaging modalities or different view modes. Detection of a multi-finger tap on the screen of the mobile communication device while viewing a first touchscreen interface for an image type invokes a second or auxiliary touchscreen interface for that image type having a subset of interface elements of the first touchscreen interface.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C. §119 from provisionalU.S. Patent Application Ser. No. 61/417,394, filed Nov. 26, 2010, thecontents of which are incorporated hereby by reference as though setforth in full.

FIELD

This patent specification relates to medical imaging. More particularly,this patent specification relates to user interfaces for medical imagereview workstations.

BACKGROUND

Substantial effort and attention has been directed to increasing thecapabilities of medical imaging systems, including continued researchand development into new medical imaging modalities, the ongoingimprovement of existing imaging modalities, and the expansion of dataprocessing, presentation, and storage capabilities for ensuring thebeneficial use of the acquired medical image data for the ultimate goalof improving overall patient health. One particularly crucial componentof the medical imaging environment is the medical image reviewworkstation, which is where all of the specially acquired and processedimage information is presented to a radiologist so that criticalhealth-related decisions can be made. As used herein, radiologistgenerically refers to a medical professional that analyzes medicalimages and makes clinical determinations therefrom, it being understoodthat such person or user of the review workstation might be titleddifferently, or might have differing qualifications, depending on thecountry or locality of their particular medical environment.

In association with the ongoing expansion of medical imaging, dataprocessing, and data storage capabilities, an ever-increasing amount ofinformation is becoming available to the radiologist at the medicalimage review workstation. Problems can arise, however, at the interfacebetween (a) the amount of information available to the radiologist, and(b) the amount of information that can be usefully accessed andperceived by the radiologist in a reasonable amount of time. Theseissues are especially important in today's radiology environment, wherethere is an ongoing tension between providing high-qualitydetection/diagnosis for each patient and maintaining adequate patientthroughput to keep costs under control. A large body of informationassociated with a patient's medical image data would have substantiallydiminished value if the radiologist does not have sufficient time,inclination, or information technology (IT) sophistication to properlyview that information. It is therefore crucial that the human-machineinterface associated with medical image review workstations be asstreamlined, appealing, and user-friendly as possible, while alsoallowing comprehensive access to the large amount of data available.

In addition to human-machine interface capability issues, the ongoingexpansion of medical imaging, data processing, and data storagecapabilities brings about problems relating to equipment acquisition,maintenance, and upgrade costs for medical image review workstations. Asknown in the art, it is often the case that medical image review for aparticular imaging modality is optimized by the use of an additionalhardware user input device other than a conventional keyboard/mousecombination, such as a specialized keypad platform having particulararrangements of buttons, knobs, sliders, joysticks, trackballs, and soforth. Although streamlining the image review process for thatparticular workstation, these specialized hardware input devices can bedisadvantageous in that they add to overall system cost, are usuallylimited to a single modality, and cannot be easily modified or upgraded.Thus, for example, if it is desired to upgrade to a new software versionhaving different workflow controls, it may be necessary to replace thespecialized keypad altogether. As another example, if it is desired toexpand the capabilities of the medical image review workstation toinclude an additional modality (for example, adding an ultrasound reviewmodality to an existing x-ray modality review workstation), then thecost and clutter of a second specialized hardware input device for thatadditional modality may become necessary.

SUMMARY

Embodiments address shortcomings of known methods and systems byproviding a user interface (UI) system for a medical image reviewworkstation that provides a streamlined, appealing, and user-friendlyexperience for the radiologist. Embodiments also provide such a UIsystem that is easily and inexpensively upgraded to accommodate newsoftware versions, new capabilities, and/or new imaging modalities forthe review workstation. Embodiments also provide such a UI system thatis readily personalized and customizable for different users at a singlereview workstation, and/or readily customizable a single radiologist atmultiple different review workstations, and allow for such a UI systemto be layered upon existing UI systems without requiring substantialchanges to current hardware configurations and without requiringsubstantial cost investment.

One embodiment is directed a computer-implemented method executed by aninterface processor and/or mobile communication device for controllingdisplay of medical images and that comprises establishing a networkconnection between a mobile communication device and the interfaceprocessor operably coupled to a review workstation operable by a user toreview medical images using a first UI. The method further comprisesdetermining a first medical image selected by the user (e.g., based onthe user highlighting or selecting an image or window containing animage). The first medical image is of a first type, e.g., one or more ofbeing an image of a first imaging modality, a first review mode (e.g., atransverse view mode for a tomosynthesis image), and generated by afirst type of imaging device. The method further comprises displaying orinvoking for display on a screen of the mobile communication device, asecond UI. The second UI comprises a first touchscreen interface forcontrolling display of the first medical image. The method furthercomprises determining a second medical image selected by the user, thesecond medical image is of a second type different than the first type,e.g., one or more of being an image of a second imaging modality, asecond review mode (e.g., a transverse mode of a magnetic resonanceimage rather than a tomosynthesis image, or a single-breast MLO view ofan x-ray image a transverse view mode for a tomosynthesis image), andgenerated by a second type of imaging device. The method furthercomprises displaying or invoking for display on the screen, a third UIthat is also a touchscreen interface but different than the firsttouchscreen interface, for controlling display of the second medicalimage.

A further embodiment is directed to a computer-implemented method forcontrolling display of medical images and that comprises displaying orinvoking for display on a screen of a mobile communication device of auser, a first touchscreen interface for controlling display of aselected medical image. The first touchscreen interface has a number ofinterface elements and is considered to be an initial or primarytouchscreen interface. The method further comprises detecting when theuser has tapped the screen with a plurality of fingers simultaneously(e.g., with a “five finger tap”). In response to detecting a five fingertap, another touch screen interface is displayed or invoked for displayon the mobile communication device screen for controlling display of theselected medical image. According to one embodiment, the secondtouchscreen interface consists of subset of the plurality of theinterface elements of the first touchscreen interface (i.e., the numberof elements of the second touchscreen interface is less than the numberof elements of the first touchscreen interface). For example, a first orprimary touchscreen interface may include interface elements for allavailable controls, whereas the second or auxiliary touchscreeninterface displayed or invoked after a simultaneous finger tap mayinclude only as many interface elements as fingers that tapped thescreen simultaneously for selected controls. For example, a firsttouchscreen UI may include 24 or other numbers of interface elements forvarious display controls, whereas a second or auxiliary touchscreeninterface displayed after a “five finger tap” includes only fiveinterface elements, e.g., five interface elements previously selected bythe user, determined to be the most popular or utilized most often, orelements positioned under fingers that tapped the screen.

Yet other embodiments are directed to computer-implemented methods forcontrolling display of medical images generated by imaging devices ofdifferent vendors or manufactures and that may involve different typesof interfaces displayed at a review workstation that receives image datafrom respective different imaging devices. For example, embodiments mayinvolve imaging devices of different manufacturers, which may be of thesame imaging modality or different imaging modalities, and that providedifferent types of interfaces for viewing medical images of the same ordifferent view modes. With embodiments, these different types interfacescan be transformed into corresponding touchscreen interfaces such that afirst touchscreen interface is generated, displayed or invoked fordisplay on a screen of a mobile communication device for a first medicalimage generated by a first imaging device manufactured or sold by afirst source, vendor or manufacturer, whereas a second, differenttouchscreen interface is generated, displayed or invoked for display fora second medical image generated by another imaging device of the samemodality (e.g., both imaging devices are tomosynthesis imaging devices),but with a different touchscreen UI. Embodiments allow users to utilizea touchscreen interface displayed on a mobile communication device tocontrol display of medical images, acquired with imaging devices of thesame or different manufacturers, which may be of the same or differentimaging modalities.

Further embodiments are directed to methods involving how a user of amobile communication device interacts with and operates touchscreen UIsgenerated according to embodiments for controlling display of medicalimages associated with respective view modes and imaging modalities.

Yet other embodiments are directed to articles of manufacture, computerprogram products and native and downloadable applications executable ona mobile communication device such as a smartphone or tablet computingdevice capable of wireless communications and configured, operable orprogrammed to execute methods according to embodiments.

For example, one embodiment is directed to a computer program product,which may comprise a non-transitory computer readable storage mediumhaving stored thereupon a sequence of instructions which, when executedby a computer or mobile communication device, perform a process forcontrolling display of medical images by displaying or invoking fordisplay on a screen of a mobile communication device a first touchscreeninterface for controlling display of the first medical image of a firsttype, and displaying or invoking for display on the screen, anothertouchscreen interface different than the first touchscreen interface forcontrolling display of the second medical image of a second typedifferent than the first type.

As another example, other embodiments are directed to articles ofmanufacture, computer program products or mobile applications which,when instructions thereof are executed, cause a computer or processingelement to perform a process for controlling display of medical imagesby detecting and responding to a multi-finger or multi-digit tap (e.g.,a “five finger tap”) by the user on a screen of the mobile communicationdevice. Thus, for example, a first touchscreen interface for controllingdisplay of a selected medical image may be displayed on the screen andincludes a plurality of interface elements, the user performs asimultaneous “multi-finger” tap, which is detected, and in response, asecond touch screen interface is generated and consists of subset of theplurality of the interface elements of the first touchscreen interface.The subset may be selected by the user to provide for customization, orbe determined based on criteria such as most frequent use or position offingers when the screen is tapped.

Embodiments may be part of or executed by a review workstation, whichmay include a non-touchscreen interface such as a keyboard and mouse,part of or executed by an interface processor operably coupled to or incommunication between a review workstation and a mobile communicationdevice, or part of (e.g., a native application) or downloaded to amobile communication device and executed by a processor or computingelement thereof. Touch screen interfaces may be displayed or invoked bycomponents of a mobile communication device and/or an interfaceprocessor operable coupled between the mobile communication device and areview workstation. Thus, an application executing on a mobilecommunication device may perform various processing to determine how atouchscreen interface should be structured and displayed.

Yet further embodiments are directed to systems configured or operableto analyze medical images using different touchscreen interfaces derivedfrom or resulting from transformation of controls of a UI of a reviewworkstation such that different touchscreen interfaces can be generatedfor different types of images, e.g., images of different imagingmodalities, different view modes, and different imaging modalities andreview modes.

For example, one embodiment is directed to a system for controllingdisplay of medical images and comprises a review workstation and aninterface processor, which may be an integral component of a reviewworkstation or a separate component that can be connected or pluggedinto the review workstation. According to one embodiment, the reviewworkstation is operated with a first UI controlled with a keyboard,mouse, trackball, joystick or other physical control element physicallymanipulated and moved by the user to select a first medical image of afirst type and control how the first medical image is displayed on ascreen of the review workstation. The interface processor, receiving ordetermining the selected first medical image and review mode and imagingmodality thereof, is configured to communicate with a mobilecommunication device, and display or invoke for display, on a screen ofthe mobile communication device, a second UI comprising a firsttouchscreen interface for controlling display of the first medicalimage. The interface processor also determines when the user hasselected another, second medical image of a second type. Selection ofanother medical image may be done through the first UI of the reviewworkstation or through the first touchscreen interface displayed on themobile communication device screen (e.g., via a toggle mechanism thatallows the user of the mobile communication device to select an imagedisplayed on the screen of the review workstation). The interfaceprocessor then displays or invokes for display on the screen, a third UIcomprising a second touchscreen interface that is different than thefirst touchscreen interface for controlling display of the secondmedical image of the second type.

System embodiments may involve or comprise only a review workstationconfigured to establish a wireless connection with a mobilecommunication device and to display or invoke display of touchscreeninterfaces, only an interface processor configured to implementembodiments, only a mobile communication device configured to implementembodiments, or a combination of components such as a review workstationand interface processor, an interface processor and mobile communicationdevice, and all of a review workstation, interface processor and mobilecommunication device.

In a single or multiple embodiments, a medical image may be selected bythe user manipulating the first UI of the review workstation ormanipulating a touchscreen interface displayed on a mobile communicationdevice, in response to which a touchscreen interface, or differenttouchscreen interface, is displayed or invoked to control display of thecurrent or selected medical image.

In a single or multiple embodiments, the interface processor and mobilecommunication device are in communication via a wireless network. Forexample, a wireless connection can be established by the mobilecommunication device being placed in communication with a wirelessaccess point that is in communication with a server hosting theinterface processor.

In a single or multiple embodiments, the interface processor, or theapplication executing on the mobile communication device, receives dataof the first UI for controlling display of the first selected image of afirst type, transforms those controls into a single-handed ordual-handed touchscreen interface for real-time control of display ofthe first selected image using the mobile communication device, receivesdata of another type of image selected by the user, and transformscontrols of the first UI into a different touchscreen interface forreal-time control of display of the other medical image using the mobilecommunication device. Touchscreen interfaces have different numbers,shapes and/or spatial arrangements of interface elements depending onthe type of image, e.g., the review mode, imaging modality and userpreferences

In a single or multiple embodiments, the medical images are presentedwithin windows of the first UI. For example, an interface with fourwindows may include four medical images, each of which is associatedwith respective view modes and/or imaging modalities. Medical images orwindows may be generated by the same review workstation, or by differentreview workstations, e.g., review workstations of differentmanufacturers, which may involve different interfaces utilized at thereview workstation. The interface processor is a part of or incommunication with the review workstation such that the interfaceprocessor determines which window is identified or selected as an activewindow, e.g., based on user manipulation of a mouse or keyboard controlat the review workstation or by use of a touchscreen interface displayedon the mobile communication device screen, to determine which medicalimage or window was selected.

In a single or multiple embodiments, the user manipulating of the mobilecommunication device itself, e.g., in the form of shaking or jigglingthe device, may be detected by an application executing on the mobilecommunication device. In response to detecting this motion, theapplication may invoke or display a different touchscreen interface fora given medical displayed, a touchscreen interface for a next image tobe analyzed is displayed, or a medical image for a new, differentpatient, e.g., a randomly selected patient or a next patient in apatient list. If the interface or patient data is not available on themobile communication device, the mobile communication device cancommunicate with the interface processor in response to detecting theshaking or jiggling motion.

Further, in a single or multiple embodiments, touchscreen interfaces canbe translated to different hands of the user. Thus, if a user is holdinga mobile communication device with a left hand such that the user'sright hand and fingers thereof are in contact with the screen, if theuser switches hands, the application detects the switch based on fingerplacement or arrangements and then displays or invokes for displayelements of the touchscreen interface that are flipped for the otherhand. Other customization features according to embodiments includetouchscreen interfaces being spatially arranged to be customized torespective lengths of respective fingers of the user of the mobilecommunication device.

In embodiments involving a multi-finger tap, the auxiliary touchscreeninterface displayed following the multi-finger tap includes only asubset of the previously displayed or primary touchscreen interface. Thenumber and/or arrangement of auxiliary interface elements may be basedat least in part upon the number of fingers that tapped the screensimultaneously, which interface elements were selected by the user, ordetermined to be utilized most often. For example, if the user tappedthe screen with five fingers (defined to include a thumb and fourfingers), then the subset consists of five interface elements, which maybe displayed at the same locations tapped by respective fingers, andthat may be spatially arranged relative to each other in the same manneras the first or primary interface. The subset of interface elements, intheir spatial arrangement, can follow the user's hand as it slides ormoves across the screen to a different screen location or as fingerlengths and/or positions are adjusted. Thus, for example, a user may beviewing a first medical image associated with a first review mode andfirst imaging modality using a first touchscreen interface (e.g., with25 or other number of interface elements), perform a five finger tap, inresponse to which an auxiliary or secondary UI with only 5 elements isdisplayed, then perform another five finger tap to toggle back to theprimary interface. Thus, a multi-finger tap can be used to switchbetween primary and secondary or auxiliary touchscreen interfaces, or toinvoke some other type of action such as switching to display of medicalimages of another patient.

Other user actions utilized by embodiments include tracking movement orpositioning of a pre-determined digit such as the user's thumb such aswhen a finger contacts or does not contact (is lifted from) the screen,to then display or invoke a new touchscreen interface, and thendisplaying or invoking the first or prior touchscreen interface when itis detected that the thumb or other pre-determined finger contacts thescreen or has returned to a pre-determined position on the screen.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an environment including components of one embodimentof a system operable to invoke touchscreen user interfaces on a screenof a mobile communication device adapted for controlling display ofdifferent types of medical images, wherein in the illustratedembodiment, an interface processor is operably coupled to a reviewworkstation and in wireless communication with a mobile communicationdevice;

FIGS. 2A-D illustrate examples of output windows of a review workstationand identification or selection of an output window as an active windowand respective touchscreen interfaces having respective interfaceelements configured according to respective spatial arrangements, anddisplayed or invoked for display on a screen of a mobile communicationdevice to control display of different types of medical images;

FIGS. 3A-B illustrate embodiments involving physical modifications topre-determined screen locations by application of one or more texturedmaterial patches to the screen at locations at which particulartouchscreen interface elements are displayed to provide haptic feedbackto the user while using the touchscreen interface, wherein FIG. 3Aillustrates textured patches applied to the screen, and FIG. 3Billustrates how the spatial arrangement of the applied textured patchesshown in FIG. 3A corresponds to particular elements of a touchscreeninterface generated according to embodiments;

FIGS. 4A-B illustrates different template patterns defining voids inrespective spatial arrangements for use in embodiments for providinghaptic feedback to a user of a mobile communication device;

FIGS. 5A-B illustrate embodiments involving physical modifications topre-determined screen locations by application templates definingrespective voids to the screen such that the voids are at locations atwhich particular touchscreen interface elements are displayed to providehaptic feedback to the user while using the touchscreen interface,wherein FIG. 5A illustrates the template shown in FIG. 4A applied to thescreen to outline groups of elements of a first touchscreen interface,and FIG. 5B illustrates the template shown in FIG. 4B applied to thescreen to outline groups of elements of a second touchscreen interface;

FIGS. 6A-D illustrate embodiments involving a multi-finger tap of ascreen of a mobile communication device by a user to invoke atouchscreen interface or a change or transformation to a touchscreeninterface, wherein FIG. 6A illustrates a first or primary touchscreenuser interface as shown in FIG. 2A and including a plurality ofinterface elements in a particular spatial arrangement and the userperforming a multi-finger tap on the screen, FIG. 6B illustrates theresulting second or auxiliary touchscreen interface following themulti-finger tap and that includes only a subset of the interfaceelements of the first or primary touchscreen user interface, FIG. 6Cillustrates the user repositioning a hand after manipulating the secondor auxiliary touchscreen user interface shown in FIG. 6B, and FIG. 6Dillustrates how the auxiliary touchscreen interface is translated fromone hand location to another location or follows the user's hand whilemaintaining the spatial arrangement of the subset of interface elementsto allow the user to manipulate the same auxiliary touchscreen interfaceat a different screen location;

FIG. 7 illustrates how embodiments may be applied to detect a change ofhand and to transform or flip a touchscreen interface configured for onehand to a configuration for an opposite hand, wherein in the illustratedembodiment, the touchscreen interface that is flipped is an auxiliarytouchscreen user interface;

FIG. 8 illustrates how embodiments may be applied to generate adual-handed touchscreen user interface;

FIG. 9 is a screenshot of a home screen displayed on a mobilecommunication device screen;

FIG. 10 is a mobile communication device screenshot that followslaunching of an application for controlling display of medical images onthe mobile communication device according to embodiments;

FIG. 11 is a mobile communication device screenshot illustrating anexample of a touchscreen user interface generally depicted in FIG. 2Aand that is generated and displayed for a first active windowcorresponding to a tomosynthesis modality;

FIG. 12 is a mobile communication device screenshot illustrating anexample of a touchscreen user interface generally depicted in FIG. 2Dand that is generated and displayed for a second active windowcorresponding to a MRI modality;

FIG. 13 is a screenshot of an auxiliary touchscreen interface generatedaccording to embodiments and resulting from the user performing amulti-finger tap on the screen of the mobile communication device whilethe touchscreen interface for the first active window corresponding to atomosynthesis modality as shown in FIG. 11 was displayed;

FIG. 14 is a screenshot of a selection or confirmation window thatallows a user to assign display controls or functions to elements of theauxiliary touchscreen interface which in the illustrated exampleincludes five interface elements as shown in FIG. 13;

FIG. 15 is a screenshot of an expanded selection or configuration windowillustrating in further detail display controls or functions that can beassigned to an element of an auxiliary touchscreen interface as shown inFIGS. 13-14;

FIG. 16 is a screenshot of an auxiliary touchscreen interface generatedaccording to embodiments and resulting from the user performing amulti-finger tap on the screen of the mobile communication device whilethe touchscreen interface for the first active window corresponding to amagnetic resonance imaging modality as shown in FIG. 12 was displayed,

FIG. 17 is a screenshot of a selection or confirmation window thatallows a user to assign display controls or functions to elements of theauxiliary touchscreen interface which in the illustrated exampleincludes five interface elements as shown in FIG. 16; and

FIG. 18 is a screenshot of an expanded selection or configuration windowillustrating in further detail display controls or functions that can beassigned to an element of an auxiliary touchscreen interface as shown inFIGS. 16-17.

DETAILED DESCRIPTION OF ILLUSTRATED EMBODIMENTS

Embodiments relate to computer-implemented methods, systems and computerprogram products or mobile applications for controlling how differenttypes medical images are displayed on a mobile communication device suchas a smartphone or tablet computing device capable of wirelesscommunications. With embodiments, a user may utilize touchscreeninterfaces that are derived from controls of user interfaces (UIs) ofknown review workstations, and which can be dynamically adapted inreal-time for controlling display of different types of images, e.g.,different types of view modes and/or imaging modalities, while the useris holding the mobile communication device. For example, in applicationsinvolving analysis of breast tissue, one touchscreen interface may bepresented for viewing an image associated with a transverse view modeand an imaging modality of magnetic resonance imaging, whereas anothertouchscreen interface is generated for viewing the same image but with adifferent view mode, or for the same view but for an image generatedusing a different imaging modality. Embodiments also accommodate medicalimages generated by imaging devices of different manufacturers andrespective different interfaces utilized to view images generated bysuch devices. Various embodiments also provide for the ability tocustomize touchscreen interfaces.

Certain embodiments are directed to methods, systems and computerprogram products or mobile applications for determining which imagestypes (of different views and/or acquired with different imagingmodalities) are selected by a user with a review workstation UI or a UIdisplayed on a mobile communication device screen, and displaying orinvoking touchscreen interfaces for respective image views and/or imagesacquired with different imaging modalities. In this manner, a user maymanipulate an existing UI of a review workstation, and also manipulatetouchscreen interfaces that are adapted or customized for differentimage views or modalities.

Certain other embodiments are directed to tapping a screen of a mobilecommunication device with multiple fingers simultaneously (e.g., withfive fingers or a thumb and four fingers) to invoke or display anauxiliary UI, which includes only a subset of interface elements, whichmay be selected by the user or selected as being utilized the mostoften, of the originally displayed UI, and they can be positioned in thesame arrangement as they were in the original UI such that positioningof fingers of the user remains unchanged. In this manner, users can“five finger tap” between the primary or complete UI and one or moreauxiliary interfaces. Screen tapping in this manner can also be used toadvance to medical images of different patients.

Referring to FIG. 1, a medical imaging environment 100 is illustratedand includes a medical image review workstation or workstation 120having an enhanced user UI including user control features implementedand/or actuated via a mobile communication device 134 such as asmartphone or tablet computing device capable of wirelesscommunications. Shown in FIG. 1 is a network 116 including a pluralityof HIS/RIS (Hospital Information System/Radiology Information System)components coupled thereto, and to which is coupled one or more imageacquisition devices examples of which include, but are not limited to, amammogram acquisition device 102, a tomosynthesis acquisition device104, an ultrasound acquisition device 106, a magnetic resonance imaging(MRI) acquisition device 108, and a generalized “other” medical imagingdevice 110 representative of, for example, one or more computerizedtomography (CT) imaging or positron emission tomography (PET)acquisition devices.

In the illustrated environment 100, a computer-aided detection (CAD)processor 112 coupled to the network 116 receives digital medical imagesfrom one or more of the devices 102, 104, 106, 108, and 110. Fortomosynthesis data sets, an additional tomosynthesis reconstructionprocessor (not shown in FIG. 1) can be coupled to the network 116 togenerate and provide a plurality of tomosynthesis reconstructed imageslices from x-ray tomosynthesis projection images provided by thetomosynthesis acquisition device 104. The CAD processor 112 processesthe medical images according to one or more CAD algorithms and providesCAD findings associated therewith. A UI implemented at the reviewworkstation 120 in conjunction with the mobile communication device 134interactively displays the medical images to a viewer or user inaccordance with one or more of the systems and methods described furtherhereinbelow. The mobile communication device 134 communicates with thereview workstation 120 by virtue of wireless communication (e.g., usingthe IEEE 802.11 “WiFi” protocol) with wireless access point 114, whichis, in turn, connected to the network 116.

Various medical images and related information are communicatedaccording to the DICOM (Digital Imaging and Communications in Medicine)standard and the network 116 supports the TCP/IP protocol, which is usedas the transport protocol for the DICOM standard. Also coupled to thenetwork 116 is a PACS archive 118, generally representing a repositoryfor medical information associated with the medical imaging environment,including both current and archived images, current and archived CADresults, radiology reports for completed cases, and so forth.Embodiments described herein can be seamlessly layered upon an existingmedical imaging workflow, in which the digital (or digitized) medicalimages are acquired, optionally processed by the CAD processor 112, anddisplayed at the review workstation 120 (optionally in conjunction withthe associated CAD results) to a radiologist, who makes a clinicaldetermination therefrom.

A UI implemented at the review workstation 120 interactively displaysthe medical images to a viewer or user (generally, “user”) ofembodiments in accordance with one or more UI programs carried out on aninterface processor 126. Included in conjunction with the UI programs onthe interface processor 126 is an auxiliary host application programthat, upon execution, communicates with the mobile communication device134 and carries out the associated functionalities described furtherherein. Included on the mobile communication device 134 is an auxiliaryremote application program that, upon execution, communicates with theauxiliary host application program on the UI processor 126 and carriesout the associated functionalities described further herein. The term“server application” can be used interchangeably with “host application”to denote described software that executes on the interface processor126.

With continuing reference to FIG. 1, a medical imaging environment 100is illustrated and includes a medical image review workstation 120having an enhanced UI including user control features implemented and/oractuated via a mobile communication device 134 such as a smartphone,tablet computing device or other mobile communication device 134.Examples of mobile communication devices 134 that may be utilized inembodiments include the IPHONE and IPAD available from Apple, Inc. andother communication or tablet computing device capable of communicatingwith a review workstation 120 or associated interface processor 126according to a preferred embodiment.

While it has been found that the IPAD 134 represents one particularlyadvantageous mobile communication device including hardware, software,network, and development platform for implementing embodiments directedto control of medical image review with UIs described further herein, itis to be appreciated that other known or hereinafter developed mobilecommunication devices 134 and platforms having generic capabilitiesanalogous to the IPAD 134 can be used in place of an IPAD 134 whileremaining within the scope of the preferred embodiments. Preferably,such other known or hereinafter developed mobile communication devices134 and platforms would include a portable, programmable touchpadcomputer having a touch-sensitive screen that is of a size and shape toaccommodate an open human hand, and would be capable of wirelesscommunication with another computer or network node using Wi-Fi,BLUETOOTH, ZIGBEE, WiMAX, Wireless USB, or any of a host of otherstandard or non-standard wireless protocols or information transfermodalities (infrared, optical, ultrasonic, etc.). While limitingmobility compared to a wireless connection, embodiments may also involvedata connectivity of the portable, programmable touchpad computerthrough a wired connection, such as wired USB, without necessarilydeparting from the scope of embodiments. Further, the size of thetouchscreen could be made smaller than that of an opened human hand,such as with the touchscreens of IPHONEs or similar portable phones,without necessarily departing from the scope of the present teachings.For ease of explanation, reference is made to an IPAD 134 as a mobilecommunication device 134 utilized in embodiments, but it will beunderstood that other mobile communication devices 134 may be utilized,and that such devices may communicate with review workstation 120 orassociated interface processor 126.

For convenience of description herein, and without loss of generality,the auxiliary host application program carried out on the UI processor126 is referred to hereinbelow as the “SVTouch host” or “SVTouch server”program. Further, for convenience and without loss of generality, theauxiliary remote application program carried out on the IPAD 134 isreferred to hereinbelow as the “SVTouch remote app.”

For purposes of clarity of description, and not by way of limitation, itis disclosed here that SVTouch™ can be seen as being a shorthand termfor SECURVIEWTouch™. SECUREVIEW is a registered trademark of Hologic,Inc., of Bedford Mass., where SECURVIEW proprietarily identifies ahighly successful and powerful medical image review workstationcurrently manufactured and sold by Hologic, Inc., the assignee of thepresent application. SECURVIEW Touch™ proprietarily identifies anextension of the SECURVIEW medical image review workstation thatincludes one or more aspects of the IPAD-implemented functionalitydescribed further herein, which one or more aspects will be publicallyintroduced at the 2010 meeting of the Radiological Society of NorthAmerica in Chicago, Ill. (RSNA 2010). However, it is to be appreciatedthat these particular monikers are not used by way of limitation, andthat the disclosed systems and methods are applicable across a widevariety of different medical image review workstations from a widevariety of manufacturers, and are further applicable across a widevariety of different touchpad-type platforms other than the IPAD 134.

Review workstation 120 implements an interactive UI using a diagnosticdisplay 122 including first and second display monitors 122 a and 122 b,an administrative display 124, and user input or control devicesincluding a keyboard 128, a mouse 132, and an application-specifichardware auxiliary input device 130, such as a workflow keypad providedin conjunction with a SECURVIEW medical image review workstation. Whenthe SVTouch host and SVTouch remote applications are activated, the IPAD134 also becomes part of the UI provided by the review workstation 120.Advantageously, the described SVTouch functionalities can be provided asan add-on that operates side-by-side with the auxiliary hardware inputdevice 130, or alternatively the described SVTouch functionalities canbe used to altogether replace the auxiliary input device 130.

Administrative display 124 is used for input and output of a widevariety of information that may be associated with a particular set ofmedical images (e.g., listings, tables, plots, text descriptions, etc.),as well as for system installation, maintenance, updating, and relatedtasks. Often provided on the diagnostic display 122 at any particulartime during case review by a radiologist are one or more diagnosticimages displayed in one or more output windows A, B, and C.

It is to be appreciated that although one or more aspects of thepreferred embodiments are described in the particular context of x-raymammography or x-ray tomosynthesis for single-modality operation, andthe contexts of various combinations of x-ray mammography, x-raytomosynthesis, ultrasound, and MRI for multi-modality operation,embodiments described herein are applicable for a variety of medicalimaging modalities in a wide variety of single-modality implementationsor multi-modality combinations, such modalities including, but notlimited to, two-dimensional x-ray, x-ray tomosynthesis, ultrasound, MRI,CT imaging, PET, single-photon emission computed tomography (SPECT), aswell as less conventional medical imaging modalities such asthermography, electrical conductivity-based modalities, and the like.Likewise, although one or more aspects of the preferred embodiments aredescribed in the particular context of breast imaging, the scope of thepresent teachings extends to medical imaging of any part of the humanbody including, but not limited to, the prostate, kidneys, otherinternal organs, head, teeth, neck, abdomen, arms, and other body parts.Examples of medical imaging systems and environments within which one ormore aspects of the preferred embodiments are applicable, which includessystems and environments having CAD capability as well as those nothaving CAD capability, can be found in U.S. Pat. Nos. 6,901,156;7,406,150; 7,809,175; 7,828,733 and U.S. Publication Nos. 2006/09885;2008/125643; 2008/0240533 and 2010/260316, each of which is incorporatedby reference herein.

Notably, the medical imaging environment 100 of FIG. 1 is presented byway of example only and is not intended to limit the scope of thepreferred embodiments to this particular scenario. By way of example,different combinations of the devices 102-132 of FIG. 1 can be placedadjacently to each other or integrated into the same hardware boxeswithout departing from the scope of the preferred embodiments. By way ofstill further example, the network 116 can be a wide-area network withthe different nodes being distributed throughout a city, a country, orthe world. Alternatively, and by way of still further example, some orall of the transfer of digital information among devices 102-132 can beachieved by physical transfer of disks, memory sticks, or other digitalmedia devices without departing from the scope of the preferredembodiments. In view of the present disclosure, a person skilled in theart would be able to implement methods, systems, and/or computer programproducts capable of achieving the described UIs and processingfunctionalities without undue experimentation, using publicly availableprogramming tools and software development platforms. By way of example,the SVTouch remote application can be developed using the Objective-Cprogramming language, while the SVTouch host program can be developedusing the C# (also termed C-Sharp) programming language.

FIGS. 2A-D illustrate how embodiments are operable transform reviewworkstation 120 UIs into touchscreen user interfaces for a mobilecommunication device to control display or invoke display of differenttypes of medical images, e.g., medical images of different types ofimaging modalities, different types of view modes, or different types ofimaging modalities and different types of view modes. Further,embodiments may be operable to control different types of images in thatthey are generated by the same imaging device, which may generatemedical images of the same imaging modality but different review modes,different imaging modalities but the same mode, or different imagingmodalities and different respective view modes. Embodiments may also beoperable to control display of different types of images generated byimaging devices of different manufacturers, and which may involvedifferent UIs at the review workstation 120.

In the embodiment illustrated in FIGS. 2A-D, the IPAD 134 and the outputdisplay 122 of the review workstation 120 while SVTouch host and SVTouchremote applications are running according to a preferred embodiment. Theoutput display 122 currently shows four output windows A, B, C, and D,which either correspond to different types of medical images in thatthey are of different modalities, or different review modes within aparticular modality. As used herein, “active output window” refers tothe particular one of the output windows A, B, C, or D to which theradiologist (“user”) is focusing their attention, identified orselected. There will generally be no more than one active output windowat any particular time.

The identity or selection of the active output window can be establishedby a variety of different methods, for example, associating the activewindow with the current location of a cursor 202 or other control or UIelement of the review workstation (which does not involve a touchscreenUS as in a UI of an IPAD). Thus, when the user hovers the cursor 202over the “A” output window, such as by controlled movement of the mouse132, then the “A” window is identified or selected as the active outputwindow. In other embodiments, it can be required that the user providesa mouse click within that output window to establish that output windowas the active window, as opposed to merely hovering the mouse over thatoutput window. Alternatively, the active window can be detected byautomated head movement detection or automated eye movement detection asused in heads-up displays for fighter aircraft or driver eye activitymonitors in newer luxury cars. Any of a variety of other means (directkeyboard input, foot petals, audio inputs) can be used to establish thecurrently active window without departing from the scope of the presentteachings.

As illustrated in FIG. 2A, when output window “A” is the active outputwindow on the output display 122, this is recognized by the SVTouch hostapplication on the UI processor 126, which then communicates with theSVTouch remote application on the IPAD 134 to invoke a first touchpadcontrol scheme 204A for that type of medical image. The first touchpadcontrol scheme 204A is specially configured to correspond to theparticular type, e.g., modality and/or review mode, of the active outputwindow “A”, including a variety of visible touchpad controls A1-A24 asshown. In this manner, the UI elements and controls provided thereby ofthe review workstation 120 are transformed into touchpad or touchscreenelements or controls for that medical image type to provide the same orsimilar controls with a different arrangement of touchscreen UI elementspresented on an IPAD 134.

As used herein, the terms “interface element,” “touchscreen element” and“touchpad control” refer to any actuable user control input provided bythe IPAD 134, and can include visible touchpad controls and non-visibletouchpad controls. As used herein, a visible touchpad control is onethat is evidenced by a viewable image of a touchpad key, knob, roller,slider, toggle switch, trackball, or the like that can be actuated atits displayed location on the touchscreen by a “virtual” pressing,turning, sliding, rolling, etc. on the touchpad. The visible touchpadcontrols A1-A6 can be simple softbuttons, for example, while visibletouchpad control A24 can be a slider control, for example. As usedherein, non-visible touchpad controls are ones that do not have aparticular location identified on the screen, but that are actuable by auser behavior, such as a single-finger linear swipe, double-fingerlinear swipe, circular swipe, double-finger separation swipe, and soforth.

According to a preferred embodiment, the SVTouch host application on theUI processor 126 and the SVTouch remote application on the IPAD 134maintain a real-time communication therebetween, and operate andcooperate such that the IPAD 134 provides a current, up-to-date touchpadcontrol scheme that corresponds in real time to the currently activeoutput window on the output display 122. Thus, when the cursor is movedto output window “B”, a second touchpad control scheme 204B for thattype of image is provided. When the cursor is moved to output window“C”, a third touchpad control scheme 204C for that type of image isprovided. When the cursor is moved to output window “D”, a fourthtouchpad control scheme 204D is provided for that type of medical image,and so forth. Advantageously, the user is automatically provided with acontrol device that is optimized for the particular output window andcorresponding type of image upon which they are focusing their attentionat that moment. In addition to being advantageous when changing focusfrom one modality type to another on the user display 122 (e.g., from atomosynthesis window to an ultrasound window), it can also beadvantageous when changing focus from one particular view type toanother particular view type within any particular modality (e.g., froma tomosynthesis projection-view window to a tomosynthesis reconstructedslice-view window), because the touchpad control scheme can be optimizedand streamlined for each particular type of viewing mode. Thus, theSVTouch host and remote applications according to the preferredembodiments herein are just as advantageous in the context ofsingle-modality review workstations as for multi-modality reviewworkstations.

Generally speaking, at least as a starting point prior to customization,the touchpad controls provided in any particular touchpad control scheme204A-204D correspond to controls that would be provided by anapplication specific hardware device, such as the application-specifichardware auxiliary input device 130 shown in FIG. 1, supra.Advantageously, however, a much richer variety of controls, essentiallya limitless variety, is made possible. A wide variety of default touchcontrol schemes can be provided in software, hardware, or firmware formsfrom which the user, or system administrator, can pick and choose atSVTouch setup time. Preferably, the touchpad control schemes can becustomized by the user, both (i) in terms of the way any particularcontrol is placed and/or actuated on the IPAD 134 and (ii) in terms ofthe corresponding function that is achieved in the active output windowof the review workstation 120 upon actuation of that control.

FIGS. 3A-3B illustrate physical modification of an IPAD to furtheroptimize user experience according to a preferred embodiment, whereintextured material patches 355, which are reasonably transparent tolight, are physically placed on the touchscreen at locationscorresponding to selected “home” touchpad controls. A haptic or tactilefeedback sensation is provided when the user places their finger on the“home” touchpad controls, providing similar advantages to the “home”keys (F and J) on a mechanical QWERTY keyboard. The user can thendevelop a muscle memory (or, alternatively, preserve a muscle memoryalready developed on another mechanical or electronic input device) forthose home keys and the relative locations of the other touchpadcontrols. The haptic feedback can prevent the need for the user to keeplooking at the IPAD 134 (and away from the diagnostic output display122) to see where the visible touchpad controls are located. Thetextured material patches 355 can be permanently affixed to the IPAD 134screen or, more preferably, can be removably affixed thereto (e.g. usingadhesives similar to those of POST-IT flags), or affixed to a screenprotector or template that is itself removable from the IPAD, or createdby making mechanical deformations in a removable screen protector.

FIG. 4A illustrates a customized template 402 comprising a rigidmaterial, such as PLEXIGLASS, into which void patterns 404 are formed inan arrangement corresponding to the touchscreen control scheme 204A ofFIG. 2A, supra. FIG. 4B illustrates a similarly customized template 406having void patterns 408 corresponding to the touchscreen control scheme204C of FIG. 2C, supra. FIGS. 5A and 5B illustrate the customizedtemplates 402 and 406 as placed on the IPAD 134 while it is providingtouchscreen control schemes 204A and 204C, respectively. The preferredembodiment of FIGS. 4A-5B, which can be used separately from or inconjunction with the preferred embodiment of FIGS. 3A-3B, provideshaptic or tactile input to the user hand to guide them to the correcttouchpad control locations, thus reducing the need to look at the IPAD134. Non-rigid materials, such as MYLAR, paper, cardboard, foam, etc.,can alternatively be used for the templates 402 and 406. The templatesmay be permanent or removable depending on the number of “layouts” oftouchpad control schemes required.

FIGS. 6A-6D illustrate a hand-specific, location-specific dynamictouchpad control scheme according to a preferred embodiment. Asillustrated in FIGS. 6A-6B, the touchpad control scheme 204A isconfigured such that, when a user taps the IPAD screen with the tips ofall five fingers simultaneously, an auxiliary touchpad control scheme654A is actuated in which a preselected subset of visible touchpadcontrols 660 (in this example, touchpad controls A6, A22, A3, A14, andA10 for the thumb, index finger, middle finger, ring finger, and littlefinger, respectively) are automatically and instantaneously placed atthe detected locations of the five fingertips.

Additionally, as illustrated in FIGS. 6C-6D, the preselected subset ofvisible touchpad controls 660 will follow the position of the hand ifthe user moves the location of their hand. In contradistinction to theone-size-fits-all mechanical device 130 of FIG. 1, the SVTouch systemincluding the customizable selection of the touchpad controls 660advantageously represents an easy to implement, cost effective solutionfor allowing a radiologist to customize the control inputs accordingtheir own personal proclivities. For example, a first radiologist mayhave a proclivity to perform a lot of on-screen measurements, and so canselect most of the touchpad controls 660 to be measurement tools or toinstantiate measurement tools. A second radiologist may have aninclination perform a lot of on-screen magnifications and gammaadjustments, and so can select most of the touchpad controls 660 to beon-screen magnifications and gamma adjustment tools.

In other preferred embodiment (not shown), the auxiliary touchpad scheme654A is configured to sense a lifting and dropping of the thumb whilethe other four fingers remain in contact with the screen, whereupon afunctionality similar to “ALT” or “SHIFT” on a QWERTY keyboard isprovided in which a secondary set of preselected visible touchpadcontrols automatically replace the preselected subset 660. The user canthen lift and drop their thumb again to bring back the originalpreselected subset 660. Other suitable finger movements to actuate andde-actuate the “SHIFT” functionality can be used without departing fromthe scope of the present teachings.

Preferably, the operation of the active-window-driven selection of thecurrent touchpad control scheme, as presented in FIGS. 2A-2D supra,continues working in conjunction with the hand-specific,location-specific dynamic touchpad control scheme of FIGS. 6A-6D toadapt to different image types. Thus, for example, if the user hastapped the touchpad at FIG. 6A and the scheme 654A of FIG. 6B is beingshown for that image type, and then the active window on the outputdisplay 122 is changed from the “A” window of FIG. 2A to the “C” windowof FIG. 2C, then the five buttons appearing under the user's fingers inFIG. 6B will automatically change to five preselected “C” buttons (forexample, buttons C4, C2, C13, C15, and C10) (not shown) for that imagetype, or that the user has previously preselected.

FIGS. 7-8 conceptually illustrate different options for customizationand response with respect to different user hands. The SVTouch remoteapp software is configured to recognize, such as by virtue of fingertiplocation pattern, whether it is the right hand or the left hand that istouching the touchpad screen. For the option of FIG. 7, the same sets oftouchpad controls are used for the opposing hands, but appear inopposing spatial order. For the option of FIG. 8, entirely differentsubsets of touchpad controls are used for the right and left hands.

FIGS. 9-17 illustrate screen shots of how embodiments may be implementedwith regard to active windows corresponding to tomosynthesis and MRIimaging modalities, and associated auxiliary touchscreen interfacesassociated with different types of images associated with theirrespective different imaging modalities.

FIG. 9 illustrates a home screen 900 of an IPAD upon which has beenloaded the SVTouch remote app, which appears as an SVTouch program icon902 just like any other application on the IPAD. In one embodiment ofcommercial operation, the SVTouch app will be downloadable from awebsite or app store such as Apple's App Store. FIGS. 10-13 and FIG. 16illustrate screenshots from an IPAD running an SVTouch remoteapplication according to a preferred embodiment, while FIGS. 14-15 andFIGS. 17-18 illustrate screenshots from a processing unit associatedwith a medical image review workstation that is running an SVTouch hostapplication according to a preferred embodiment.

As made apparent by FIG. 9, the IPAD 134 can belong the user in theirpersonal capacity and can be used for a variety of different purposes orendeavors, such as listening to music, surfing the web, doing onlinebanking, and so forth. The IPAD 134 does not need to have any particularacquisition-related association with any particular medical image reviewworkstation or other aspect of the hospital information system, butrather simply needs to have the SVTouch application loaded onto it,placed in communication with the wireless access point 114, and providedwith the IP address of the server that is running the SVTouch hostapplication. Thus, advantageously, the same IPAD used for SVTouchfunctionality can be used to provide many conveniences and/or comfortsfor the radiologist as well, such as allowing them to bring music fromhome, use headphones, make telephone calls, check their e-mail, surf theweb, control the ambient room lighting (using an IPAD app), and talk byintercom to other radiologists, technologists, or hospital personnel.

As illustrated in FIG. 10, upon invoking the SVTouch remote app, theuser is presented with an introductory screen 1000 providing a listing1002 of review workstations having an SVTouch server (SVTouch host)application to which they can connect, which can be one of manydifferent workstations (e.g., their clinic office, their hospitaloffice, their residence or vacation home office, etc.) with which theyare associated.

FIGS. 11 and 12 illustrated how embodiments can generate differenttouchscreen interfaces for display on a mobile communication device fordifferent types of images, which may involve one or more of differenttypes of imaging modalities, different view modes, and different imagesgenerated by different imaging device manufacturers. FIG. 11 illustratesan example of a touchpad control scheme 1104A for a first type ofmedical image or an active output window corresponding to an imagingmodality type such as a tomosynthesis modality, analogous to thetouchpad control scheme 204A of FIG. 2A. FIG. 12 illustrates an exampleof a touchpad control scheme 1204D for a second type of medical image ora second active output window corresponding to a different imagingmodality type such as a MRI modality, analogous to the touchpad controlscheme 204D of FIG. 2D. Shown as persistent controls in the upper righthand corner of the touchpad screen are a cursor control toggle 1106 anda microphone on/off toggle 1108. When the cursor control toggle 1106 isset in an active state, the IPAD touchscreen (or, alternatively, someportion of the IPAD touchscreen) becomes operative as a regular touchpadmouse for controlling the cursor 202 (see FIGS. 2A-2D) on theworkstation display 122 to select a different window or type of image.When the cursor control toggle 1106 is returned to an inactive state,the IPAD is returned to a mode in which the SVTouch functionalitiesdescribed herein are resumed.

FIG. 13 illustrates an auxiliary touchpad control scheme 1354A thatappears upon tapping the touchpad control scheme of FIG. 11(tomosynthesis) with all five right-hand fingers simultaneously,analogous to the functionality described in FIGS. 6A-6D above, andfeaturing the five visible touchpad controls 1360 a-1360 e as shown fora particular image type.

FIGS. 14-15 illustrate a 5 button configuration window of the SVTouchhost application for selecting the five “favorite” touchpad controlsthat will appear in FIG. 13. As illustrated, a set of touchpad controlselection windows 1360 a′-1360 e′ are provided for allowing the user tochoose the functionalities associated with the visible touchpad controls1360 a-1360 e, respectively, of FIG. 13.

Also provided are a set of selection windows 1470 for allowing the userto choose the functionalities associated with the non-visible touchpadcontrols, including a 1-finger swipe left, 1-finger swipe right,2-finger swipe left, and 2-finger swipe right. Shown in FIG. 15 is oneof the selection windows expanded to show a listing 1580 oftomosynthesis-centric workstation functions from which the user canselect to associate with the respective “touchpad controls. The columnlabeled “A” in FIGS. 14-15 is for designating the primary five favoritetouchpad controls, while the column labeled “B” in FIGS. 14-15 is fordesignating the secondary five favorite touchpad controls that willappear in the “SHIFT” or “ALT” scenario described above (e.g., when theuser lifts and drops their thumb while the other four fingers remain incontact with the screen).

FIG. 16 illustrates an auxiliary touchpad control scheme that appearsupon tapping the touchpad control scheme of FIG. 12 (MRI) with all fiveright-hand fingers simultaneously, analogous to the functionalitydescribed in FIGS. 6A-6D above, and featuring the five visible touchpadcontrols 1660 a-1660 e as shown. FIGS. 17-18 illustrate the 5 buttonconfiguration window of the SVTouch host application for selecting thefive “favorite” touchpad controls that will appear in FIG. 16. Includedin FIGS. 17-18 are a set of touchpad control selection windows 1660a′-1660 e′ corresponding respectively to the assignments for the visibletouchpad controls 1660 a-1660 e. Further included in FIGS. 17-18 is aset of selection windows 1770 for the non-visible touchpad controlfunctional assignments, and a listing 1880 of MRI-centric workstationfunctions from which the user can select to associate with therespective touchpad controls.

Notably, although the listings 1580 and 1880 of workstation functionfrom which the user can select in FIG. 15 (tomosynthesis imaging type)and FIG. 18 (MRI imaging type) number in the dozens, there can moregenerally be hundreds of different review workstation function optionsprovided. Optionally, audio controls (“next CAD mark”, “next slab”) canbe associated with visible or non-visible touchpad controls, such thatthose commands can be invoked either by the required touch/gesture or bya voice command input. According to one preferred embodiment, a macrolanguage is provided so that touchpad controls can be programmed toinvoke specialized routines on the review workstation 120. By way ofexample, the macro language can provide the ability to specifyparticular cursor movements (in absolute screen coordinates and/oroffset coordinates relative to the currently active window), cursorbutton clicks, particular keystroke sequences, and so forth.

By way of example, one particular macro can be typed in as the followingASCII sequence: (@100,@300,2){$L1} {#250} {100,450} {$L2} hj. Themeaning of this ASCII sequences means: move to absolute position(100,300) on screen 2, then click left mouse button, then wait 250milliseconds, then move to relative position (100,450), then click theright mouse button, then type the characters “hj”.

For one preferred embodiment, a macro recording capability can also beprovided that allows the user to record and store particular sequencesof mouse movements, keyboard inputs, mouse clicks, and the like, tocreate a macro without needing to code it in the macro programminglanguage. For one preferred embodiment, the macro programming,recording, and storage methodologies can be similar to those of apopular and easy to use Windows-based macro system called AutoHotKeys,information about which can be found on the World Wide Web atautohotkeys dot com (www.autohotkeys.com).

There is further disclosed the following concepts, which areintrinsically provided as part of the above described technology orwhich can be readily integrated therewith or incorporated in conjunctiontherewith. The IPAD/SVTouch can be used as a UI device for a reviewworkstation such as a Hologic SECUREVIEW DX, optionally to completelyreplace the SECURVIEW DX workflow keypad, to replace the mouse, theon-screen buttons, and/or to replace even the keyboard (since the IPADon-screen keyboard is available.) The IPAD/SVTouch can be placed into anadjustable orientation for best use as a touch-screen controller, with abalance of ergonomics with visibility of the screen at the most commonviewing angles. Soft wrist supports can be provided with theIPAD/SVTouch, and registered indents or touch tape used to allow theuser to find a “home” position akin to the little nubs on QWERTYkeyboard F and J keys. The IPAD/SVTouch can be customized by touchingeach finger to the IPAD to determine the length of fingers' reach,optionally to scale the location of control buttons to the length ofreach, optionally to scale the size of control buttons according tolength of reach, and optionally to scale the length of sliders. Seamlessleft hand versus right hand control can be provided, such as byoptionally auto-flipping the presentation of visible touchscreencontrols when the hand is changed.

Optionally, various functions of the administrative display 124 of FIG.1 can be incorporated onto the IPAD/SVTouch display, such as the patientworklist, or optionally all of the non-imaging administration-typeinformation can be displayed on the IPAD/SVTouch such that all the otherscreens are used exclusively for imaging information. The IPAD/SVTouchcan be used to display RIS information, user documentation, and trainingmaterials. Tabbed controls can be provided for access to thekeypad/RIS/worklist/reporting system.

Optionally, a non-diagnostic quality miniaturized version of a medicalimage with CAD markers located thereon can be displayed on theIPAD/SVTouch (i.e., an annotation road map), and the CAD markers can bechecked off by touching them at their locations on the touchscreen. TheIPAD/SVTouch can be used as a UI for clinical studies, the touch screenfor recording finding locations on low-resolution images.

The IPAD/SVTouch can be used to replace existing keypad controls and toprovide new controls for displaying and viewing different types ofmedical images. For binary controls, the assignment of any SECURVIEWfunction to an IPAD button can be provided. The IPAD/SVTouch can usesliders and regional XY touch-sensitive areas for 1D and 2D controls,and optionally facilitate separation of some 2D controls into two 1Dcontrols (such as WW/WC adjustment). The IPAD/SVTouch can use 2Dmulti-touch to control zooming, 1D multi-touch controls for stretching1D things like contrast range or brightness range (setting simultaneousupper and lower ranges), slide controls to slide through temporal seriesof prior studies, multi-touch to control the two ends of cut-planes inreconstructed MR (and other) images in 3D data sets, and drag and dropcontrols (e.g., drag from a navigator on the IPAD to window icons on theIPAD rather than, or in addition to, moving the mouse on the workstationdisplay screen). The IPAD/SVTouch can incorporate connections toSECURVIEW DX such as wired USB for direct control, and wireless (802.11a/b/g/n) for wireless control, and wireless (BLUETOOTH) for proximitycontrol (the ability to walk up to the SECURVIEW with an IPAD 134 and itconnects automatically). That way doctors can bring their ownIPAD/SVTouch and configure them to work with any appropriately equippedSECURVIEW DX.

Although particular embodiments have been shown and described, it shouldbe understood that the above discussion is not intended to limit thescope of these embodiments. While embodiments and variations of the manyaspects of the invention have been disclosed and described herein, suchdisclosure is provided for purposes of explanation and illustrationonly. Thus, various changes and modifications may be made withoutdeparting from the scope of the claims, and many alterations andmodifications of the present invention will no doubt become apparent toa person of ordinary skill in the art after having read the foregoingdescription. Thus, it is to be understood that the particularembodiments shown and described by way of illustration are in no wayintended to be considered limiting.

For example, while SVTouch remote app configuration associated with FIG.1 is described as being placed in communication with the wireless accesspoint 114 and being provided with the IP address of the server that isrunning the SVTouch host application, in other embodiments there can beprovided automated or semi-automated proximity-sensing functionality,wherein the IPAD 134 can automatically recognize that it is next to areview workstation, using BLUETOOTH or infrared communications, forexample, and can automatically connect to that review workstation andinstantiate an SVTouch control session. By way of further example,non-visible touchpad controls that can be incorporated into theIPAD/SVTouch can include a gesture-based language, similar to theGraffiti single-stroke shorthand handwriting recognition system used inpersonal digital assistant devices based on the PALM OS.

By way of still further example, it is to be appreciated that thefunctions of the SVTouch host (also called SVTouch server) program canbe segregated and placed on different pieces of computing equipment. Forexample, there can be provided a centralized SVTouch registration andcustomization service run on a single computer in a HIS/RIS network, andthis central server can feed the required information to the SVTouchhost and remote apps at the beginning of each radiology review session.Alternatively, there can be a web-based SVTouch registration andcustomization service hosted by an ASP (application service provider) sothat individual hospital IT departments do not need to worry aboutproviding it, or other cloud-based or cloud-like implementation can beprovided.

By way of even further example, within the scope of the preferredembodiments is to harness the accelerometer function of the IPAD toassist guiding the workflow. For example, the user can shake the IPAD toselect a random patient from the unread patient worklist to read next,jiggle the IPAD to the left to return to the previous patient, jigglethe IPAD to the right to proceed to the next patient, and so on.

By way of still further example, it is to be appreciated that themultiple output windows A-D with different types of images as shown inFIGS. 2A-2D supra can further be differentiated based on systemcomponent manufacturer in addition to modality type and review modetype. Thus, for example, the review workstation 120 could be providedwith the ability to run review workstation packages from two differentmanufacturers (e.g., GE and Hologic), with the user being able todisplay one output window from Hologic and another output window fromGE, both windows being of the same modality (such as conventionalmammography) and of a similar review modes (such as single-view CCimages). In such case, the IPAD/SVTouch would be configured to present afirst touchpad control scheme optimized for a first type of medicalimage, e.g., the GE mammo/single view CC window when the GE outputwindow is active, and then a second touchpad control scheme optimizedfor a second type of medical image, e.g., the Hologic mammo/single viewCC window when the Hologic output window is active.

As a further example, different imaging devices may generate differentmedical images of the same imaging modalities, and different UIs at thereview workstation for those different imaging devices can betransformed into respective touchscreen UIs for display on a mobilecommunication device. Thus, a first touchscreen UI may be generated,displayed or invoked for display for a tomosynthesis image generatedwith a first type of imaging device or a tomosynthesis imaging device ofa first manufacturer or vendor, whereas a second touchscreen UI may begenerated, displayed or invoked for display for another tomosynthesisimage generated with a second type of imaging device or tomosynthesisimaging device of a second manufacturer or vendor.

Therefore, reference to the details of the preferred embodiments are notintended to limit their scope, and it will be understood that reviewworkstation UIs may be transformed into touchscreen UIs generated,displayed and invoked for display for different types of imagingmodalities, different types of view modes, different types of imagingmodalities and view modes, and images generated using different types ofimaging devices, e.g., imaging devices of the different manufacturers,which may be of the same or different imaging modality.

Further, it will be understood that embodiments may be directed tocomputer-implemented methods, involving and/or performed by a reviewworkstation, interface processor and mobile communication device,systems, and non-transitory computer program products, articles ofmanufacture or mobile applications, including native and downloadableapplications executed on a mobile communication device, and that suchprograms, instructions or applications may be stored in memory includingone or more of cache, RAM, ROM, SRAM, DRAM, RDRAM, EEPROM and othertypes of volatile or non-volatile memory capable of storing data, andthat a processor unit that executes instructions may be or includemultiple processors, a single threaded processor, a multi-threadedprocessor, a multi-core processor, or other type of processor capable ofprocessing data. Method embodiments may also be embodied in, or readablefrom, a computer-readable medium or carrier, e.g., one or more of thefixed and/or removable data storage data devices and/or datacommunications devices connected to a computer. Carriers may be, forexample, magnetic storage medium, optical storage medium andmagneto-optical storage medium. Examples of carriers include, but arenot limited to, a floppy diskette, a memory stick or a flash drive,CD-R, CD-RW, CD-ROM, DVD-R, DVD-RW, or other carrier now known or laterdeveloped capable of storing data. The processor executes programinstructions within memory and/or embodied on the carrier to implementmethod embodiments. Further, embodiments may reside and execute on amobile communication device such as a Smartphone, tablet computingdevice and other mobile communication devices.

Additionally, it will be understood that the interface processor may bea stand-alone component or integrated into a review workstation, andthat embodiments may be executed by the host program, by the remoteprogram or application, or by both. Thus, it will be understood thatembodiments may involve displaying touchscreen interfaces or invoking orproviding touchscreen data to a mobile communication device to beprocessed and displayed on a screen of the mobile communication device.Thus, a touchscreen interface may be displayed or invoked by display bythe mobile communication device and/or interface processor.

Further, while certain embodiments are described with reference to a“five finger tap” embodiments may involve other numbers of fingers(defined to include fingers and thumb) simultaneously tapping an IPADscreen. Further, such tap functions may be utilized to display or invokean auxiliary display and/or other functions such as switching to adifferent patient. Moreover, the subset of interface elements displayedupon detecting a multi-finger tap may be selected by the user or beselected or identified (e.g., by the host or remote program orapplication) based at least in part upon interface elements determinedto be the most popular or utilized most often and elements positionedunder fingers that tapped the screen.

For example, a multi-finger tap with fingers arranged over a first setof keys would result in a subset of interface elements including onlythose tapped interface elements, whereas a multi-finger tap with fingersarranged over a second, different set of keys would result in a subsetof interface elements including only those tapped interface elements. Itwill also be understood that different subsets of UI elements displayedin response to an action such as a 5 finger tap can be displayed orinvoked for display depending on the type of the currently displayedimage, such that the subset of interface elements are adapted to andsuitable for the particular image type being displayed before the actionoccurs.

Additionally, while certain embodiments are described with reference totouchscreen interfaces, review modes and imaging modalities associatedwith breast tissue, it will be understood that embodiments may apply tomedical imaging of various other parts of the human body.

Certain embodiments are described with reference to medical images beingselected by a user, but medical images may also be automaticallyselected, e.g., based on a sequence or alphabetical listing.

While this specification describes certain embodiments individually,embodiments may also involve various combinations of features, forexample, a combination displaying or invoking for display touchscreeninterfaces in combination with one or more or all of the “five fingertap” functionality, selection of an image or active window using thereview workstation or mobile communication device, patches or templatesto provide haptic feedback, macro execution, customizing location ofinterface elements displayed according to the positioning and length ofa user's fingers, translating or flipping touchscreen interfaces for useby different hands, generating a single handed or dual handedtouchscreen interface on a mobile communication device, detectingshaking or jiggling to invoke some action such as switching fromreviewing images of one patient to reviewing medical images of anotherpatient, and allowing for control of review of images generated bydifferent review workstation manufacturers using a mobile communicationdevice. Thus, embodiments may involve any one of these or otherembodiments or aspects thereof individually or in combination, anddescription of an embodiment or feature thereof individually is notintended to limit the scope of combinations of such embodiments orfeatures.

Moreover, where computer-implemented methods and associated user actionsare described in a particular sequence to indicate certain eventsoccurring in certain order, those of ordinary skill in the art havingthe benefit of this disclosure would recognize that the ordering may bemodified and that such modifications are in accordance with thevariations of the invention. Additionally, parts of methods may beperformed concurrently in a parallel process when possible, as well asperformed sequentially.

1-19. (canceled)
 20. A computer-implemented method for controllingdisplay of medical images on a medical image review workstation using amobile communication device operatively coupled to the workstation, themethod comprising: the mobile communication device displaying orinvoking for display on a screen thereof a user interface forcontrolling display of a medical image on the review workstation, theuser interface comprising a plurality of user interface elementsdisplayed on the mobile communication device; the mobile communicationdevice detecting selection of a user interface element of the displayedplurality; the mobile communication device sending an instruction to thereview workstation to cause a change in the display of the medical imagebased on the selected user interface element.
 21. The method of claim20, wherein the user interface is customizable to control display ofmedical images generated by different imaging modalities.
 22. The methodof claim 20, wherein the user interface is customizable to controldisplay of medical images generated by different view modes.
 23. Themethod of claim 20, wherein the user interface is customizable tocontrol display of medical images generated by different imagingmodalities and view modes.
 24. The method of claim 20, wherein the userinterface is customizable to control display of medical images generatedby different imaging devices.
 25. The method of claim 24, wherein thedifferent imaging devices include a first imaging device made by a firstmanufacturer, and a second imaging device made by a second manufacturerdifferent from the first manufacturer.
 26. The method of claim 24,wherein the different imaging devices include a first imaging devicehaving a first interface type displayed on the workstation, and a secondimaging device having a second interface type displayed on theworkstation, the first interface type being different from the secondinterface type.
 27. The method of claim 20, further comprising themobile communication device automatically customizing the user interfacebased on the displayed medical image.
 28. The method of claim 27,wherein the user interface is automatically customized by the mobilecommunication device based on a view mode of the displayed medicalimage.
 29. The method of claim 27, wherein the user interface isautomatically customized by the mobile communication device based on animaging modality of the displayed medical image.
 30. The method of claim27, wherein automatically customizing the user interface compriseschanging the plurality of interface elements displayed on the mobilecommunication device.
 31. The method of claim 30, wherein changing theplurality of user interface elements displayed on the mobilecommunication device comprises changing a characteristic selected fromthe group consisting of number, shape and spatial arrangement of thedisplayed user interface elements.
 32. The method of claim 27, furthercomprising the mobile communication device automatically customizing theuser interface in real time based on the displayed medical image.
 33. Acomputer program product comprising a non-transitory computer readablestorage medium having stored thereupon a sequence of instructions which,when executed by a computer, causes the computer to perform a processfor controlling display of medical images on a review workstation usinga mobile communication device operatively coupled to the reviewworkstation, the process comprising: the mobile communication devicedisplaying or invoking for display, on a screen thereof, a userinterface for controlling display of a medical image on the reviewworkstation, the user interface comprising a plurality of user interfaceelements displayed on the mobile communication device; the mobilecommunication device detecting selection of a user interface element ofthe displayed plurality; the mobile communication device sending aninstruction to the review workstation, the instruction to cause a changein the display of the medical image based on the selected user interfaceelement.